
2 8 MAR W. 



WIPO 



PA 1292831 




miffliulmmiuiluiiilBfufflmimiiiinmniimiimuiiiiiiuiui 




ninttilrn 



■ CS OMOBi tt 



UNITED STATES DEPARTMENT OF COMMERCE 



United States Patent and Trademark Office 

March 08, 2005 

THIS IS TO CERTIFY THAT ANNEXED HERETO IS A TRUE COPY FROM 
THE RECORDS OF THE UNITED STATES PATENT AND TRADEMARK 
OFFICE OF THOSE PAPERS OF THE BELOW IDENTIFIED PATENT 
APPLICATION THAT MET THE REQUIREMENTS TO BE GRANTED A 
FILING DATE UNDER 35 USC 111. 

APPLICATION NUMBER: 60/557,224 ^ 
FILING DATE: March 29, 2004 



PRIORITY 
DOCUMENT 

SUBMITTED OR TRANSMITTED IN 
COMPLIANCE WITH RULE 17.1(a) OR (b) 



nA/Y-V 




»,♦«%. ' ' ' 




By Authority of the 

COMMISSIONER OF PATENTS AND TRADEMARKS 



% 




P. $WAIN 
Certifying Officer 





iM(»»iMitinwihti)iiitHt)iaifhiiiiHiniiiiimtmi ^ 



immuniimiupimmmimuiiiuumtiuimuiimnnininmmmiffniinflmiTiuiiii 




Please type a plus sign (+) Inside this box ^ j "H 

PTO/SB/16 (02-01) 
Approved for use through 10/31/2002. OMB 0651-0032 
Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE 
Under the Paperwork Reduction Act of 1 995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. 

PROVISIONAL APPLICATION FOR PATENT COVER SHEET 

This is a request for filing a PROVISIONAL APPLICATION FOR PATENT under 37 CFR 1 .53 (c). 



fry | Express Mall Label No. EV 312 069 295 



Date of Deposit: March 29, 2004 



INVENTOR(S) \ 


Given Name (first and middle [if any]) 


Family Name or Surname 


Residence I 
(City and either State or Foreign Country) . I 


SANDEEP 
YIBIN 


DALAL 
YANG 


Cortlandt Manor, NY I 
Pine Brook, NJ I 



.CM 
CO CM 

LO 



EO 

■CM 

IO 



CO 



□ Additional inventors are being named on the 



separately numbered sheets attached hereto 



TITLE OF THE INVENTION (280 characters max) 



VIDEO QUALTIY ENHANCEMENT AND/OR ARTIFACT REDUCTION USING CODING INFORMATION FROM A 
COMPRESSED BITSTREAM 



CORRESPONDENCE ADDRESS 

Direct all correspondence to: 

IS Customer Number 24737 ^ 
OR *" Type Customer Number here 



*24737* 

IIIIIIIHIIIIIUIIIIIIIII 



Firm or 

Individual Name 



Address 



Address 



City 



Country 



U. S. PHILIPS CORPORATION 



P.O. BOX 3001 



BRIARCLIFF MANOR 



USA 



State 



Telephone 



NY 



(914) 945-6000 



ZIP 



10510 



Fax I (914)332-0615 



ENCLOSED APPLICATION PARTS (check all that apply) 



Specification Number of Pages 
X Drawing(s) Number of Sheets 



13 



I I CD(s), Number 
Other (specify) 



I I Application Data Sheet. See 37 CFR 1 .76 



METHOD OF PAYMENT OF FILING FEES FOR THIS PROVISIONAL APPLICATION FOR PATENT (check one) 



I I Applicant claims small entity status. See 37 CFR 1.27. 

I I A check or money order is enclosed to cover the filing fees s 



The Commissioner is hereby authorized to charge filing 
fees or credit any overpayment to Deposit Account Number: 

I I Payment by credit card. Form PTO-2038 is attached. 



14-1270 



FILING FEE 
AMOUNT ($) 

160.00 



The invention was made by an agency of the United States Government or under a contract with an agency of 
the United States Government. 

No. 

□ Yes, the name of the U.S. Government agency and the Government contract number are: . 



Respectfully submitted, 
SIGNATURE 




TYPED or PRINTED NAME ERIC M. BRAM 

TELEPHONE (914) 333-9635 



Date March 29, 2004 

REGISTRATION NO.: 37,285 
(if appropriate) 

Docket Number: 



US 040162 



USE ONLY FOR FILING A PROVISIONAL APPLICATION FOR PATENT 

This collection of information Is required by 37 CFR 1.51. The information Is used by the public to file (and by the PTO to process) a provisional application. 
Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1 .14. This collection is estimated to take 8 hours to comptele, including gathering, preparing, and submitting 
the complete provisional application to the PTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this 
form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, 
Washington. D.C.. 20231 . DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO; Box Provisional Application, Assistant Commissioner for 
Patents, Alexandria. VA 22313. 



Codv provided bv USPTO from the IFW Imaqe Database on 03/07/2005 



PROVISIONAL APPLICA TION COVER SHEET 

Additional Page 

PTO/SB/16(02-01) 
Approved for use through 10/31/2002. OMB 0651-0032 
Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE 
Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless It displays a valid OMB control number. 



| Docket Number 


Type a plus sign <+) 
US040162 inside this box — > + 


INVENTOR(S)/APPLICANT(S) 


Given Name (first and middle [if any]) 


Family or Surname 


Residence 

(City and either State or Foreign Country) j 


LILLA 

CORNELIS CONRADUS A. M. 

• 


BOROCZKY 
VANZON 


MOUNT KISCO, NY 
COLD SPRING, NY 



Number 2 of 2 



WARNING: Information on this form may become public. Credit card information should not 
be included on this form. Provide credit card information and authorization on PTO-2038. 



Codv provided bv USPTO from the IFW Imaqe Database on 03/07/2005 



US040162 

1 

VIDEO QUALITY ENHANCEMENT AND/OR ARTIFACT REDUCTION 
USING CODING INFORMATION FROM A COMPRESSED BITSTREAM 

5 

This application claims priority under 35 U.S.C. § 120 and 35 USC § 365(c) to 
International Patent Application serial number IB2003/0057 filed on December 4, 2003 
and entitled "A Unified Metric For Digital Video Processing (UMDVP) " to Boroczky, et 
al; to International Patent Application Number IB2003/0055 filed November 28, 2003 and 
10 entitled "System and Method for Joint Video Enhancement and Artifact Reduction for 

Coded Digital Video", to Boroczky, et al.; and to U.S. Patent Application 10/029,828. All 
of these applications are incorporated herein by reference in their entirety as if folly set 
forth herein. 

1 5 This invention pertains to the digital video signal processing, and in particular, to a 

system and method of enhancing video quality and/or reducing coding artifacts in a 
decoded video signal by using coding data from the coded video signal. 

Moving Picture Expert Group (MPEG) video compression, or coding, technology 
20 facilitates many current and emerging video products (e.g., DVD players, high definition 
television decoders, and video conferencing) by requiring less storage and less bandwidth. 
However, it is well known that such lossy compression technology (MPEG-1, MPEG-2, 
MPEG-4, H.26x, etc.) can cause the introduction of coding artifacts and decrease picture 
quality of the video signal. 
25 Sometimes low bit rates are chosen to achieve bandwidth efficiency. The lower the 

bit rates, the more objectionable become the impairments introduced by the compression 
encoding and decoding processing. For example, for digital terrestrial television 
broadcasting of standard-definition video, a bit rate of around 6 Mbit/s is considered a 
good compromise between picture quality and transmission bandwidth efficiency (see P.N. 
30 Tudor, "MPEG-2 Video Compressions," IEEE Electronics & Communication Engineering 
Journal, December 1995, pp. 257-264). However, broadcasters often broadcast digital 
video signals having bit rates far lower than 6 Mbit/s so that they can provide more 
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programs per digital television channel. So artifacts are generated and picture quality is 
reduced. 

» 

Therefore, recently, as discussed in International Patent Application Number 
IB2003/0055 filed November 28, 2003 and entitled "System and Method for Joint Video 
5 Enhancement and Artifact Reduction for Coded Digital Video," video display devices have 
been provided with systems and methods for post-processing the decoded video signal for 
display to enhance the picture quality and/or reduce the video artifacts. 

Meanwhile, digital video sources such as DVDs, video tapes, broadcast signals, etc. 
often provide coded digital video signals that are digitally compressed or coded in a 
1 0 different video format than the video format with which they will be displayed. For 
example, motion pictures/film sequences are typically encoded or compressed in the 
progressive scan format at a frame rate of 24 frames/second that has been used for decades. 
On the other hand, many video display devices such as television receivers display video 
signals using an interlaced video display format of 50 or 60 fields per second. 
15 FIG. 1 shows an exemplary block diagram illustrating a system and method of 

video signal enhancement and/or artifact reduction using coding information from a coded 
digital video signal. The coded digital video signal may have been encoded using MPEG- 
1, MPEG-2, MPEG-4, H.26x, or any other desired video-encoding standard. For ease of 
explanation, hereafter it will be assumed that the encoded digital video signal has been 
20 encoded according to the MPEG-2 standard. Furthermore, in the example of FIG. 1, the 
encoded digital video signal represents a film sequence that has been encoded at a source 
frame rate of 24 frames/second. 

In a block 1 1 0, a video decoder decodes the encoded, or compressed, digital video 
signal. The block 110 provides the appropriate decoding according to whatever standard 
25 has been used for coding the encoded digital video signal. The block 110 outputs a 

decoded digital video signal having a video source format (e.g., progressive scan at 24 
frames/second). The operation and construction of such a video decoder would be well 
understood to those skilled in the art. 

Then, in block 120, a format converter converts the video source format of the 
30 decoded digital video signal to a suitable video display format for display by a particular 
television receiver or display device. In the example of FIG. 1, a standard 3:2 pulldown 
algorithm is employed to convert the decoded digital video signal from a video source 
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format that is progressive scan at 24 frames/second, to a video display format that is 
interlaced at approximately 60 fields/second. Alternatively, the display format may be 
interlaced at approximately 50 fields/second, in which case a 2:2 pulldown is performed 
instead. The block 120 outputs a format-converted decoded digital video signal having a 
5 video display format (e.g., interlaced at 60 fields/second). Again, the operation and 
construction of such a format converter implementing a 3:2 or 2:2 pulldown algorithm 
would be well understood to those skilled in the art. 

Meanwhile, in block 130, a video encoding metric calculation module calculates 
one or more video encoding metric(s) for the encoded digital video signal using coding 

10 information and/or other features of the encoded digital video signal. For example, when 
the encoded digital video signal is an MPEG-2 video data stream, such coding information 
may include a quantization parameter (q_scale), the number of bits used to code a 
luminance block (num_bits), etc. Such coding information may be provided within the 
bitstream of the encoded digital video signal. Exemplary algorithms are described, for 

1 5 example, in International Patent Application serial number EB2003/0057 filed on 
December 4, 2003 and entitled "A Unified Metric For Digital Video Processing 
(TJMDVP) " The block 130 outputs the video encoding metric(s) corresponding to the 
decoded digital video signal having the video source format (e.g., 24 frames/sec. for film 
sequences). 

20 Next, in block 140, a video encoding metric conversion module converts the video 

encoding metric information output by block 130 into a format corresponding to the 
format-converted decoded digital video signal output by the format converter in block 120. 
That is, the block 140: receives at its input the outputs of the video encoding metric(s) 
corresponding to the decoded digital video signal having the video source format (e.g., 24 

25 frames/sec. for film sequences); executes a conversion algorithm on the received video 
encoding metric(s); and outputs converted video encoding metric(s) corresponding to the 
format-converted decoded digital video signal output by the block 120 having the video 
display format (e.g., interlaced at approximately 60 fields/second. 

Finally, in block 150, the converted video encoding metric(s) are employed in an 

30 algorithm to enhance the picture quality (e.g., sharpness enhancement, resolution 

enhancement, etc.) and/or to reduce video compression artifacts in the format-converted 
decoded digital video signal. 
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Accordingly, the system and method described above can enhance the quality 
and/or reduce video artifacts in a video signal after it is decoded and prior to display on a 
display device. 

However, such a system and method suffers from several disadvantages. 

5 Converting the video encoding metric(s) so they can be applied to the format-converted 
decoded digital video signal having the video display format is computationally 
burdensome. Also, artifact reduction algorithms tend to look for artifacts amongst spatially 
neighboring pixels. It is rather complex to apply these algorithms to interlaced video data. 
Furthermore, the quality of the results produced by the picture quality enhancement and/or 

10 video compression artifact reduction algorithm(s) is impaired. Assumptions must be made 
regarding how video encoding metric(s) for a progressive scan signal at a frame rate of 24 
frames/second are applied, e.g., to fields of an interlaced signal at a video display rate of 50 
or 60 fields/second. 

* 

Accordingly, it would be desirable to provide an improved system and method of 
1 5 video quality enhancement and/or artifact reduction using one or more video encoding 

metric(s) derived from coding information and/or other features of an encoded digital video 
signal. The present invention is directed to addressing one or more of these concerns. 

In one aspect of the invention, a method of processing a digital video signal 
comprises decoding an encoded digital video signal to produce a decoded digital video 
20 signal having a progressive scan format at a frame rate of approximately 24 frames/second; 
calculating at least one video encoding metric from the encoded digital video signal; 
executing a video quality improvement algorithm on the decoded digital video signal 
having the progressive scan format at the frame rate of approximately 24 frames/second, 
using the calculated video encoding metric, to produce a processed decoded digital video 
25 signal having the progressive scan format at the frame rate of approximately 24 

frames/second; and converting the processed decoded digital video signal from the 
progressive scan format at the frame rate of approximately 24 frames/second format to an 
interlaced format at one of approximately 50 fields/second or approximately 60 
fields/second. 

30 In another aspect of the invention, a method of processing a digital video signal for 

display on a display device comprises decoding an encoded digital video signal to produce 
a decoded digital video signal having a video source format; calculating at least one video 
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encoding metric from the encoded digital video signal; executing a video quality 
improvement algorithm on the decoded digital video signal having the video source format, 
using the calculated video encoding metric, to produce a processed decoded digital video 
signal having the video source format; and converting the processed decoded digital video 
5 signal from the video source format to a video display format suitable for display on the 
display device. 

In yet another aspect of the invention, a system for processing a digital video signal 
for display on a display device comprises a decoder for decoding an encoded digital video 
signal to produce a decoded digital video signal at a source frame rate; a video encoding 

10 metric calculation module for calculating a video encoding metric from the encoded digital 
video signal; a post-processor for executing a video quality improvement algorithm on the 
decoded digital video signal at the source frame rate, using the calculated video encoding 
metric, to produce a processed decoded digital video signal; and a format converter for 
converting the processed decoded video signal from the source frame rate to a display 

1 5 frame rate suitable for display on the display device. 
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FIG. 1 shows an exemplary block diagram illustrating a system and method of 
video signal enhancement and/or artifact reduction using coding information from a coded 
video signal; 

5 FIG. 2 shows an exemplary block diagram illustrating a system and method of 

video signal enhancement and/or artifact reduction using coding information from a coded 
video signal according to one or more aspects of the present invention. 

10 FIG. 2 shows an exemplary block diagram illustrating a system and method of 

video signal enhancement and/or artifact reduction using coding information from a coded 
video signal according to one or more aspects of the present invention. 

The coded digital video signal may have been encoded using MPEG-1, MPEG-2, 
MPEG-4, H.26x, or any other desired video-encoding standard. For ease of explanation, 

15 hereafter it will be assumed that the encoded video signal has been encoded according to 
the MPEG-2 standard. Furthermore, in the example of FIG. 2, the digital video signal 
represents a film sequence that has been encoded at a source frame rate of 24 
frames/second. 

In a block 210, a video decoder decodes the encoded, or compressed, digital video 
20 signal. The block 2 1 0 provides the appropriate decoding according to whatever standard 
has been used for coding the digital video signal. The block 210 outputs a decoded digital 
video signal having a video source format (e.g., progressive scan at 24 frames/second). 
The operation and construction of such a video decoder would be well understood to those 
skilled in the art. 

25 Meanwhile, in block 220, a video encoding metric calculation module calculates 

one or more video encoding metric(s) for the encoded digital video signal using coding 
information and/or other features of the encoded digital video signal. For example, when 
the encoded digital video signal is an MPEG-2 video data stream, such coding information 
may include a quantization parameter (q_scale), the number of bits used to code a 

30 luminance block (numjDits), etc. Such coding information may be provided within the 
bitstrearn of the encoded digital video signal. Exemplary algorithms are described, for 
example, in International Patent Application serial number IB2003/0057 filed on 
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December 4, 2003 and entitled "A Unified Metric For Digital Video Processing 
(UMDVP)." The block 220 outputs the video encoding metric(s) corresponding to the 
decoded digital video signal having the video source format (e.g., 24 frames/sec. for film 
sequences). 

5 Then, in block 230, a post-processor uses the calculated video encoding metric(s) to 

enhance the picture quality (e.g., sharpness enhancement, resolution enhancement, etc.) 
and/or to reduce video compression artifacts in the decoded digital video signal by 
employing the metric(s) in one or more picture quality enhancement and/or video 
compression artifact reduction algorithm(s). Advantageously, the post-processor 230 

10 operates upon the decoded digital video signal while it is still in the native video source 
format. Accordingly, identifying artifacts amongst spatially neighboring pixels is 
simplified. Furthermore, the quality of the results produced by the picture quality 
enhancement and/or video compression artifact reduction algorithm(s) is improved since, 
for example, no assumptions are necessary regarding how metric(s) obtained for a 

1 5 progressive scan signal at a frame rate of 24 frames/second are applied, e.g., to fields of an 
interlaced signal at a video display rate of 50 or 60 fields/second. 

Finally, in block 240, a format converter converts the video source format of the 
processed decoded digital video signal to a suitable video display format for display by a 
particular television receiver or display device. In the example of FIG. 2, a standard 3:2 

20 pulldown algorithm is employed to convert the processed decoded digital video signal from 
a video source format that is progressive scan at 24 frames/second, to a video display 
format that is interlaced at approximately 60 fields/second. Alternatively, the display 
format may be interlaced at approximately 50 fields/second, in which case a 2:2 pulldown 
is performed instead. The block 240 outputs a format-converted processed decoded digital 

25 video signal having a video display format (e.g., interlaced at approximately 60 

fields/second or approximately 50 fields/second). Again, the operation and construction of 
such a format converter implementing a 3:2 or 2:2 pulldown algorithm would be well 
understood to those skilled in the art. 

Accordingly, the system and method described above can enhance the quality 

30 and/or reduce video artifacts in a video signal after it is decoded and prior to display on a 
display device. Among other things, the computational burden is substantially reduced, 
compared to the system and method illustrated in FIG. 1. 
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While embodiments are disclosed herein, many variations are possible which 
remain within the concept and scope of the invention. Such variations would become clear 
to one of ordinary skill in the art after inspection of the specification, drawings and claims 
herein. The invention therefore is not to be restricted except within the spirit and scope of 
5 the appended claims. 
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CLAIMS: 



1. A method of processing a digital video signal, comprising: 

decoding (210) an encoded digital video signal to produce a decoded digital video 
signal having a progressive scan format at a frame rate of approximately24 frames/second; 
calculating (220) at least one video encoding metric from the encoded digital video 

signal; 

executing a video quality improvement algorithm (230) on the decoded digital 
video signal having the progressive scan format at the frame rate of approximately 24 
frames/second using the calculated video encoding metric, to produce a processed decoded 
digital video signal having the progressive scan format at the frame rate of approximately 
24 frames/second; and 

converting (240) the processed decoded digital video signal from the progressive 
scan format at the frame rate of approximately 24 frames/second format to an interlaced 
format at one of approximately 50 fields/second or approximately 60 fields/second. 

2. The method of claim 1, wherein calculating (220) at least one video encoding 
metric includes calculating a Unified Metric For Digital Video Processing (UMDVP) 
value. 

3. The method of claim 1, wherein the video encoding metric is calculated using at 
least one of a quantization parameter or a number of bits employed to code a luminance 
block of the coded digital video signal. 

4. The method of claim 1, wherein converting (240) the processed decoded video 
signal from the progressive scan format at the frame rate of approximately 24 
frames/second format to an interlaced format at approximately 60 fields/second comprises 
executing a 3:2 pulldown algorithm. 



Codv provided bv USPTO from the IFW lmaqe Database on 03/07/2005 



4 



US040162 



10 

5. The method of claim 1, wherein converting (240) the processed decoded video 
signal from the progressive scan format at the frame rate of approximately 24 
frames/second format to an interlaced format at approximately 50 fields/second comprises 
executing a 2:2 pulldown algorithm. 

6. A method of processing a digital video signal for display on a display device, 
comprising: 

decoding (210) an encoded digital video signal to produce a decoded digital video 
signal having a video source format; 

calculating (220) at least one video encoding metric from the encoded digital video 

signal; 

executing a video quality improvement algorithm (230) on the decoded digital 
video signal having the video source format using the calculated video encoding metric, to 
produce a processed decoded digital video signal having the video source format; and 

converting (240) the processed decoded digital video signal from the video source 
format to a video display format suitable for display on the display device. 

7. The method of claim 6, where the video source format is progressive scanned at 
approximately 24 frames/second. 

8. The method of claim 6, where the video display format is interlaced at 
approximately 60 fields/second. 

9. The method of claim 6, where the video display format is interlaced at 
approximately 50 fields/second. 

10. The method of claim 6, wherein calculating (220) at least one video encoding 
metric includes calculating a Unified Metric For Digital Video Processing (UMDVP) 
value. 
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1 1 . The method of claim 6, wherein the video encoding metric is calculated using 
at least one of a quantization parameter or a number of bits employed to code a luminance 
block of the coded digital video signal. 

12. The method of claim 6, wherein converting (240) the processed decoded video 
signal from the video source format to the video display format comprises executing a 3:2 
pulldown algorithm. 

■ » 

13. The method of claim 6, wherein converting (240) the processed decoded video 
signal from the video source format to the video display format comprises executing a 2:2 
pulldown algorithm. 

14. A system for processing a digital video signal for display on a display device, 
comprising: 

a decoder (210) for decoding an encoded digital video signal to produce a decoded 
digital video signal at a source frame rate; 

a video encoding metric calculation module (220) for calculating a video encoding 
metric from the encoded digital video signal; 

a post-processor (230) for executing a video quality improvement algorithm on the 
decoded digital video signal at the source frame rate using the calculated video encoding 
metric to produce a processed decoded digital video signal; and 

a format converter (240) for converting the processed decoded video signal from 
the source frame rate to a display frame rate suitable for display on the display device. 

15. The system of claim 14, where the video source format is progressive scanned 
at approximately 24 frames/second. 

i 

16. The system of claim 14, where the video display format is interlaced at one of 
approximately 50 fields/second or approximately 60 fields/second. 

17. The system of claim 14, wherein video encoding metric calculation module 
(220) calculates a Unified Metric For Digital Video Processing (UMDVP) value. 
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18. The system of claim 14, wherein the extracted coding information includes at 
least one of a quantization parameter or a number of bits employed to code a luminance 
block of the coded digital video signal. 

19. The system of claim 14, wherein the format converter (240) executes a 3:2 
pulldown algorithm. 

20. The system of claim 14, wherein the format converter (240) executes a 2:2 
pulldown algorithm. 
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ABSTRACT 

5 A system and method of processing a digital video signal for display on a display 

device, includes decoding an encoded digital video signal to produce a decoded digital 
video signal having a video source format; extracting coding information from the encoded 
digital video signal; executing a video quality improvement algorithm on the decoded 
digital video signal having the video source format using the extracted coding information, 
10 to produce a processed decoded digital video signal having the video source format; and 
converting the processed decoded digital video signal from the video source format to a 
video display format suitahle for display on the display device. The system and method 
enhance the quality and/or reduce video artifacts in a video signal after it is decoded and 
prior to display on a display device. 
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